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Abstract 

This approach to dynamic scene analysis is a qualitative one. It 
computes relative depths using very general rules. The depths calcu- 
lated are qualitative in the sense that the only information obtained is 
which object is in front of which others. The motion is qualitative in 
the sense that the only required motion data is whether objects are 
moving toward or away from the camera. Reasoning, which takes 
into account the temporal character of the data and the scene, is 
qualitative. This approach to dynamic scene analysis can tolerate 
imprecise data because in dynamic scenes the data are redundant. 

Keywords: qualitative vision, dynamic scene analysis, rtlativelrur** 
facendepths- 

1 Motivation^ for qualitative vision 

For many reasons computer vision has proven a difficult task, far 
more difficult than was originally suspected. The complexity of the 
real world .3 sampled spatially and f emporaily and projected onto 
a time ordered seo/ience of frames. The description of objects, re- 
lationships and events among those objects is a signal to symbol 
transformation which requires the top-down use of knowledge (i.e. 
an interface to a/memory). Because the projection process *!oees* a 
dimension, mtetpretation must be able to tolerate ambiguous data. 
Noise compounds the ambiguity of the frame sequence. Roberts’ 
work in this 0eid, 22\ attempted to compensate for noise using 
several heuristics for line detection and a top-down modeJ-fittmg ap- 
proach. An approach to solving this problem of model-fitting under 
noise is given by Brooks 8j. In this work, he has the problem of 
unknown transforms between model and image; to solve this prob- 
lem he uses a constraint manipulation package to limit the matches 
between image and model and hypothesize other matches consistent 
with the constraints. The transforms are initially under- specified, 
then increasingly constrained. 

Two approaches have proven to be particularly restrictive. The 
drat is the to cus on single frame analysis. Eariy researchers ieit that 
it was necessary to first process one frame, and only then examine 
the subsequent frame. This is a sterile approach because it avoids 
all temporally changing scenes (i.e., things like pictures and maps), 
including most scenes of interest. The second approach which has dis- 
appointed is the careful computation of numerical features in a data 
driven manner. Examples are 3-D positions of feature points ob- 
tained via structure- from-motioo or of surface normals from optical 
Sow, shape from shading, or texture, motion parameters: v„,v v ,v ft , 
and optimization of objective functions (for citations describing these 
various approaches see: i’, 20], 24l, 27], 32 ). Most rely on an in- 
verse transformation, from two dimensions to three, and that, com- 


bined with the noise inherent in sensor and the sampling and digitiz- 
ing processes, means that algorithms providing quantitative solution 
will be inherently very sensitive to noise. 

Along with the growing interest in dynamic scenes, the realiza- 
tion that improving accuracy in a highly restricted set of features 
ao * particularly help the interpretation process some vision re- 
searchers 25] are being drawn to the qualitative approaches being 
used for common sense reasoning, naive physics and circuit analy- 
sis r 6j, [12]. The r e asons for this are that qualitative approaches 
show that it is possible to obtain useful results when solving prob- 
lems with uncertain, approximate or only signs of parameters. The 
representations of the problem domains are an attempt to capture 
the fundamental nature of the system, while avoiding the complexity 
of dynamic equations. Currently, much of the work done in qualW 
tative physics involves determining appropriate states and symbols 
and an understanding of the nature of state change. Another im- 
portant component is a simulation process, which allows one to get 
a grasp on causality. The qualitative approaches thus far attempted 
in A1 have generally included things like signs of derivatives [9) [16- 
or transitions ’10]. Computer vision is also a testbed where consid- 
erable intelligence is required. Further, the data in computer vision 
are always noisy, frequently redundant, and often misleading. 

Another approach for handling the noise for bottom- up processes, 
having support from biological vision systems, is to use a variety of 
window sizes or a collection of band- passed images. Larger sized 
operators average over a greater area, and thus ibr reasonably well 
behaved noise, the noise has less effect on the result. Unfortunately, 
the larger the window, the more likely it is computing a single result 
over two or more different pixel source populations. Some researchers 
have proposed using a set of different window sues, large ones for 
large scale and perhaps low contrast changes and smaller ones for 
more local changes 17], the scale space is a continuous vermioa of 
this t 32j. It is oot dear how to combine information among these 
many channels, partly because the channels are being used for two 
different :;ungs: detecting .o r measuring) at different scales, ana 
using larger channels to reduce nowe effects at the lower channels. 

Event detection in its most general sense locates the interface 
between qualitatively different sources of pixel population. The idea 
of event detection is * ot to smooth over noise, and thus over differ- 
ent pixel populations by using a magically chosen window size, but 
instead to detect where the pixel population changes and avoid any 
integration across that boundary. Using this paradigm are : 5j, '111. 
and [13]; also t 24j, using a finite element approach, can fracture the 
surface at appropriate places. 

This now* issue has especially frustrated dynamic scene researchers 
because it has been shown mathematically that ail 3-D information 
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(to a scale factor) U available in the optical flow field. Attempts 
to get the in format too have been fruitless because even the best ob- 
tainable flow fields are too badly corrupted. Thompson, et al., [261 
take the approach that if precise values are not computable, then 
compute the qualitative information: which segment is the occluder 
and which is the occluded. Jain [14] has also obtained this infor- 
mation for different sorts of scenes. Both use only a crude, though 
computable, approximation to optical Sow. The first uses an ap- 
proximation to the flow field called a disparity field requiring good 
feature detection and correspondence algorithms. The second uses a 
more qualitative approximation, computing the time history of pixel 
changes. But in any case it is clear that useful results are possible, 
even from the noisy data available, using and computing qualitative 
attributes rather than precise, brittle ones. 

2 Using models in computer vision 

There are a number of reasons for building or using a model in com- 
puter vision. 

1. A model provides a simplified representation. For example, the 
motion of a point may be specified with initial slate and state 
transition equations, thus it is not necessary to store, for each 
point m time, the position of a point. 

2. At some, perhaps iow. level, the data can be said to be under- 

stood when there is a model which Sts them. For example, at 
a very low level when the data St, e g., a straight line, the line 
is a model for the data, and as a line is understood, so 3 the 
underlying phenomenon giving rise to the data. This is like 
number l above At a higher level, for the model which is a 
line y - one can say e g., m is velocity and k is starting 

position At the highest level, if the model is a frame with slots, 
then the general practice is to use a prion default values for 
slots which are not filled from the data. Thus, a limited data 
set which cause a particular frame to be instantiated, triggers 
a top-down use of the model in which more is understood than 
can be derived n an immediate sense from the data. 

3. if the data can De said to fit some model, then, because in 
general, each data point need not be kept around, the inter- 
pretation processes can be made more tolerant to noise. !n 
the representation, data can be allowed to lie within a range of 
values, as signal - noise, where the noise has some understood 
or assumed statistical properties. 

4 Missing or amoiguous data can be handled by assuming the 
data exist according to the model, but are not measurable for 
wme reason This property of models is heavily reued upon by 
computer vision researeners because occlusion is rampant. 

5. When a model .s uvaiiabie. it can play the roie of a kind of 
short term memory where the integration of lata, especially 
of errorful data can oe incorporated This property was used 
:n 2 .n their ^ery early work on iynamic scene analysis, in a 
primitive fashion 

There are a a u inner of representation techniques for models. Per- 
haps the most popular is to use frames having slots IS A abeiled 
‘r vrr.e * number >f ’abejed riots which 'an re filled with 

numbers, attributes a e. symbols), variables or unxstoot&er :rames 
The procedures which manipulate these slots, for example how the 
slot with label velocity relates to dynamical equations which can pre- 
dict future positions are also part of the modelling process The 
dynamical equations are modelling the motion These sorts of cal- 
culations are not as easily represented in frames, since this sort of 
knowledge is more naturally given as procedures Interpretation in- 
volves instantiating the model from among a set of competitors that 
best match the .lata An event indicates where models, or perhaps 
uniy parameters of the models, change Confidence is a number ex* 
pressing some kind of probability that the model is correct or that 
the data measurement s correct, when that nformatioo s* available 


3 Purpose and use of chronologies 

The very earliest works in dynamic scene analysis required the rep- 
resentation of velocities and positions over time. It is not enough to 
give a simple initial state, because most motions are not descnbable 
with simple dynamic equations (consider hierarchical or non-rigid 
motions), and because they do not incorporate changes in motion 
descriptions, and because other interesting temporal characteristics 
are not included in a natural fashion. The early works did not keep 
chronologies. Instead, in ;2] for example, they kept a model of the 
scene for one time instant only, and used that to predict the model for 
the next frame. This implicitly incorporates the initial state descrip- 
tion. Robot planning frequently requires the description of several 
actions over an extended period of time. These are generally inspired 
from the approach of describing the state of the world and robot at 
each time instant (for an advanced use of this see 7'). 

Tsotsos *28] made extensive use of chronologies which were essen- 
tially time-ordered positions of points to choose among hypotheses 
for high level motion descriptions (e.g. expand, sway). His system 
chose the best hypothesis by examining the time-course of confi- 
dences of the possible schemas. This example exemplifies a major 
use of chronologies: to disambiguate local motions into more global, 
longer term motion descriptions. Other uses are to be able to predict 
future positions and circumstances, to identify interesting motions, 
and to localise events in the motions. In addition to obtaining long 
term motion descriptions, a history of events or of motions, or of 
relationships between object parts, is useful on its own, or for deriv- 
ing ether, even higher level descriptions. That is, ne may be able 
to describe oscillatory motion as such, rather than as a repeating 
sequence of position and velocity. 

Chronologies are not really models, however, because they nei- 
ther provide a simplified representation for the data, nor do they 
provide understanding. They provide a description. Chronologies 
also provide a representation in which noise tolerance, occlusion and 
integration of ^ata in a temporal fashion can be supported, especially 
under the control of temporally dependent operation. 

4 Local Temporal Inferencing 

4.1 Introduction 

When vaJues can be tied to a number one, they are quantitative. 
Permitting bounds on values, that is, restricting them to an inter- 
val on the number line, one can still do numerical operations on 
them 3 Naive physics researchers use the qualitative (symbolic) 
descriptors: increasing or decreasing. These values are obtained by 
considering the sign of derivatives, also an interval. If the sign is 
positive, the variable values are increasing We aiao use the intervals 
I-3O.0 - ), ?0-.0M. (0~ . -oc) as qualitative values. Another sort of 
qualitative value is a relative statement. For example x is faster than 
y, or a is closer than b. This sort of relation constrains the value of 
x with respect to y (and vice versa), but loes not tie the value to 
the number line. Hasse diagrams sre a graphical representation de- 
scribing such relative statements when the relation provides a partial 
ordering The qualitative examnie involving partial order is different 
from the notions of state and of symbol. It is a rompanson. The 
ordering qualitative example :s also rrore robust to noise - though 
not Decause the error tolerance j greater 

There are other qualitative relations between attributes which 
are interval in their nature, i e . which have begin and end points. 
Vilain 30 and Allen 4, have developed an interval- based tempera: 
reasoning and labelling system Their works, and those of others, are 
applicable to domains like story understanding, wn*re there tend to 
be fixed endpoints to the temporal intervals. Vere 29 has developed 
a system which will generate parallel plans for achieving g^ls within 
time constraints. 



4.3 Constraint* on domain and general description 

For the work reported here we wish to describe the relations among 
objects, without recourse to object or scene models, over extended 
frame sequences. In particular, we with the program to provide the 
relative depths among surfaces, when computable, and histones of 
surface to surface relations. 

The data for this work are tiarordered lists of occluder-occluded 
pairs and directions of motion in depth of surfaces (toward or serif 
from camera). [14) and [26) have shown methods whereby occluder- 
occluded relations may be obtained. No further data are required. 

Suppose that all objects are stationary (i.e., as in static scene 
analysis). The data provided are triples of the sort: A occludes B In 
terms of depth, x, this means, for surfaces with changes in depth that 
are negligible with respect to tnter-eurface depths, that z(A) < z(fl). 
We use the notation A < B. Occlusion data thus places a partial 
ordering on the depths of surfaces; and for static scenes, transitivity 
suffices to provide all computable depth constraints between surface 
patches. This partial ordering does not change over time. Thus, for 
example, given the data set: A < B; B < C; 3 < D, transitivity 
gives us that: A < C, A < D, and that there a a o ordering in 
depth between C and D. Inconsistencies in data and in deductions 
are trivially detected, though not trivially resolved. 

When objects are permitted to move in a plane parallel to the 
image plane the rule for combining depth constraints is again tran- 
sitivity. If there is no change m depths of objects then the relative 
depths will jot change. 

If objects are allowed to move in depth, then the depth order 
obtained by a local occlusion analysis can no longer be used as a 
sorting criterion. Transitivity dees not hold into the future when 
depths change over time. An approach to this problem is to project 
depth constraints between two objects into the future, and then use 
those derived constraints in transitive relations at the time of inter- 
est. 

4.3 Velocity rules 

There are four pftysicaily derived rules which give the projection 
into the future for the depth orderings. Motion m depth a v. If 
Stgn\v t < 0 then motion a fowsvd the observer on the temporal 

interval [t.t - At) for 5»yniv tt *^|) > 0 motion is swsy from 
the observer on the same interval Sign[v) - 3 means there is no 
significant motion in depth The four rules are: 

• rtu'el A t *• 3, and •-tt.j.iiA) - 0 and v f = 0 ==> 

■li* ji 3t~c.t 

• rule: A, • 3, and A) ‘ Oandt'd. - 0 =» 

• rui>3 At 4 - 3 t and vtt-j^n.4) • 0 and v f > J => 

t*- ii * 

• ruie4 A# «• 3* and *■- 0 anc d 

At . ,'.i * 3, * 

These ru;es are ail expressed :n 

At r 3t and < - 0 and ' - 0 ~ === > 

A* i . ^ St i - .11 

They ve referred to n ater 'ex: vs '*iucUy ^xie$ For other motions 
of A and 3. that u. for A t 3» wnete v*Ai ► 0 or vi3) < 0 
there a no relative depth information between A and 9 at time 
f - At Inconsistencies are detectable m this scheme when conflicting 
relations are derived ; cycles are uetecteuj 

4.4 Temporally local inferencing on qualitative rela- 
tions 

The data are occluder-occiudee pairs, and the direction of motion 
in lepth -toward or away from observer) From these data, one 
can derive mfront (and behind) relations for the present tune using 
transitivity of depth ordering, and for the future using the velocity 


rut*. The derivations for the future hold under the assumption that 
there is no change in the direction of depth velocity (magnitude is 
unimportant). However, we prefer not to have to deal with such an 
unstructured, open-ended future. Since the data are arriving at this 
system at each time step, we make the inferences into the future 
for one time step only. Thus, at time to, we have a set of depth 
relations *(to)- From these relations we use the velocity rules to 
derive, for the next time tj, a set of relations 4*(t|). This set of 
relations, ignoring tine and labels, will be a subset of the relations 
at to. Recall velocity rules are of the sort A < B at to plus some 
constraints on velocity of A and B =» A < B at f|. There is no 
point in applying the transitivity rule at this point, it will not add 
any new arcs. Incorporating the data at time t| will add some new 
relations. This will give rise to the set of relations *~(t t ). Now one 
applies transitivity at this point to obtain the set ♦ (*»), and the set 
of relations is ready to project into the future ooe time step again. 
See figure 1 for a layout of the order of operations on the relations. 

Thus, this system incrementally incorporates the data as it be- 
comes available. It makes no attempt to predict further into the 
future than to the next time step. A time step is defined as when 
the next datum is available. It has no memory beyond ooe time sup. 
It is focal, temporally speaking. More global temporal knowledge is 
kept elsewhere in the system - specifically, in the object histories. 

In the system there may be several relations between a given pair 
of segments. That is, each relation has two segments aad a label. For 
the segment pair, A and B, we may have, e g. 

order label 

A. < B rule 1 

* < B data 

As long as the data are consistent and correct, these inferences 
will iteratively build a consistent partial order on the segments which 
is as complete as is possible for these rules at the current time. What 
happens when a datum is incorrect? In that case we will have an 
mconxutcat set of relations. This inconsisUncy is signaled by a cycle 
m the graph. For example, suppose we have the relatioo: .*» < 3 : 
ru/el for the graph in ♦*(<). We then read the datum 3 < A. The 
graph then contains the cycle A — 3. 

Because we have labels on relations, we know what gave rise to the 
inconsistency. For the above example we know that, because there 
is a cycle, the datum 3 < A is wrong, or the ruUl applications 

was wtong cr both. Conceivably, we could trace the cause of the 
inconsistency back further into the past. For the above example, if 
the rule l application at time t - l was wrong then either the v( A) was 
wrong, t>(3) was wrong, the relation A < 3 at time t - 1 was wrong, 
or any subset of these three was wrong. For this ooe inconsistency 
involving only two objects and two relations we have axeady fingered 
as possible culprits four attributes or relations going oack only one 
time sup. Indeed, if we kept only a slightly more compel* audit trail 
the relation A < 9 at time t - t could be further tracked damn. This 
gives rise to even more possibilities of the source of inconsistency 
even more remouiy in time. 

We are not doing this for a number of reasons. The most im- 
portant of these ss than in a dynamic scene understanding system, 
one does not have the resources to spend a lot of tune and energy 
resolving past conflict; data are continually arriving, and it is better 
to have the current land future) interpretations oe correct than those 
of the past. Secondly, many culprits are fingered for each inconsis- 
tency This is a lot of over n* ad and cannot be resolved or reduced 
fiwiu 'A* .AAntaUon currently available. For a third reason, resolu- 
tion is possible oniy in the future when more data is available - there 
■a no resolution possible in the past (where the inconnsuncy arose). 
Fourth is that we rely on the fact that there are a lot of data. Even 
though some are wrong, most will be right, we do not eant to devoU 
much effort to inconsisUncy resolution because we may expect that 
future data will set things right. There is one important consequence 
of this for the impiemralauoa we do not keep sa extensive audit 
trail. We abei each arc with only the rule that most recently derived 
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So ww do not nub any attempt to undo any bad «&cta from 
pnaaiHs bod data in the past The ptnatol data an a different matter. 
Wt do, k o a twr , want the c orract data to eventually outweigh any 
inc orrec t inferences. We hare a number of option* on h tm to go 
about doing thin Essentially there are two quest io tir 

1. bow to propagate, to the next time step a r el a tion which baa a 
contradiction 

2. bow to incorporate a contradicting pair at r el a t ione into object 
histories 

We deal with this difficulty by taking the posit i on that ooe rma 
trust the current data at the current time. Any inferences from 
that data, especially into the future may be inspect, but the data 
thsmeehrea are asumed to be correct for the time sour. We could 
take the position that any relation which contradicts dele will be 
deleted immediately, and is not allowed to propagate into the future. 
But we still have the problem oi contradictory r e lat i one which do not 
have dele on either of the labels. For example, see figure 2 in which 
we show the derivation of an inconsistency. 

We are dream to the use of a certainty factor for each relation 
in order to accomodate the possibility of occasionally invalid data. 
The certainty cf dais relations will be highest. As inferences are de- 
rived, the certainty factor of those relations will decrease. There are 
a number of technical difficulties involved in dealing with certainty 
factors and getting them to be rigorously correct. We avoid these 
by relying on the fact of large amounts of mostly right data. To rig- 
orously derive a calculus of certainty factors, it is necessary to have 
a sufficiently deep understanding of the nature of the domain, espe- 
cially of the nature of the data, noise, and sometimes even s yrien 
probability values of the data, se well as assumptions of indepen- 
dence, or and knowledge of correlations la the spirit of qualitative 
processing, we wish to avoid making such restrictive assumptions 
until necessary. In this system we are attempting a qualitative ap- 
o roach in which we know there is no we, though we don’t know its 
precise precise properties. Because of the fortunate choice of using 
dynamic scenes as data, however, we can use the fact that the data 
will be mostly redundant. Thus even though rigorous derivations for 
certainties have been done 23! and may he applicable we are not 
currently investigating that direction. We just want certainty fac- 
tors to decrease with time and with transitive "distance* from dale. 
There are a number of choices on how to combine certainty factors 
when making inferences We are currently experimenting with thw. 
The problem of which of two contradictory relations to propagate 
we deal with heuristic ally : relations which are contradictory are not 
permitted to activate the transitive rule. All other relations may 
activate both transitive and velocity rules. We put this restriction 
on tranaitive-denved contradictory rules for computational reasons 
only. Many relations are derived using transitivity, and when one 
of the links w suspect, all links derived from it are suspect. Infer- 
ences whose certainties are decreasing to zero are deleted alter a fixed 
number of time steps. 

5 Chronologies 

3.1 Representational issue — indexing 

La building a chronology of depth-ordered relations among surface 
patches for use, either as a descriptive device, or as an intermediate 
data structure for further processing, there are two ways of indexing. 
The first m to organise the relations temporally In dynamic scene 
anaiysw, unlike story understanding, the data are arriving in a tune- 
srdered fashion, e g. in frame t, the is sane set ♦(•) of relalioas. at 
frame t +• l some other set ♦{• ■+• I). The chronology of relations has 
the same appearance as the data with the addition of derived rela- 
tions In this case it is easy to see what is happening at a particular 
time instant, because time is the index into list of relations, e g 


Isms rclshMI 

1 (< A 3) (< A D) 

2 (< A 3) (< A D) 

3 (< AB) (< A D) 

4 (< AB) (< BC) (< AC) 

We sss in one indexing step which r e lations exist at t=3. Given 
tbs way the velocity rules are for mulated , it is alao saner to m a ke 
predictions into the next time step. For example, if the motions in 
depth of A, B,C are negligible, then at time 4 we can make the pre- 
diction that at tiffts 5, the following refetions will hold: (< A B), (< 
BC),(<AC). 

The second indexing -'thod is to organise by relation. That is, 
ooe surface is tbs primary index, the second surface the secondary 
index. For example: 

1* 2* temporal tnlcrvel* 

A B (1, sow) 

C (4, sow) 

O (1,3) 

B C (4, sow) 

If relations persist, or are repetitious, then it saves on space to 
index by surfaces. This re pres e ntation trades off r elationsh ip storage 
for temporal storage space advantageously when r el a t i ons are long- 
lived or recur frequently. To determine at a particular time instant 
which relations are active requires inspection of a lot of data. The 
time course of relations is rawer to. access. Event marking makes de- 
riving an interval- b as e d description easy. And this method of index- 
ing is better for dealing with noise removal, occlusion and integration 
over time. 

5.3 History and world model of depths 

Despite the fact that dynamic vision has a lot of data available, 
thanks to its ramps*! redssdascy [31 j, it is both more efficient as 
well as satisfying to keep histones of relations which are indexed by 
surface. The fact remains, however, that in order to make derivations 
(or predictions) using the velocity rules and to be ai. 'e to make a 
computation ally fast statement about the relative depths at time 
sow, we keep the current List of relations, though redundant with 
histones. That is, for now we have a "temporally indexed* set of 
relations. For now as well as all the past we have object- indexed 
relations. This means that if relative depths for any past time is 
desired, though the information is calculable (indeed, was calculated, 
then discarded), from the histories, it is not immediate. Rather, the 
system will have to step through the histories, essentially re-creating 
the world for the desired time. There are certain sumlarities with 
mvuwiiMf 9j. For sow •» rmn get ad ordered relative depth map 
by doing a straight-forward topological sort [15]. 

6 Experiments 

The local temporal Lnferencmg system was implemented as described 
in a previous sect ion. We made a few adjustments, for pruning pur- 
pose, to the lnferencmg procedure as follows. 

• Relations whiefl were contradictory, e g. A < 3 and B < A, 
were not permitted to participate in any transitivity inferences. 
This is because the only result from applying transitivity on 
contradictory relations m many more contradictory relations. 
Contradictory relations are allowed to propagate into the future 
with the velocity rules. 

• Only relations derived from transitivity which had a larger or 
equal confidence factors than other relations already present 
(between the same nodes) were posted. For example, suppose 
we have the relation A < 9 with confidence TO present, then 
we derive A < 3 with confidence 35 from transitivity. That 
new relation is ignored. 
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• Data nUlioM in take* with cmMan ID (indicated mc/ = 
ID), and other nUtiov already p Meant bet wee n the same 
aodae ae the data relation were deleted. That is, if we have 
A < B becanae of a r%Ut application with c/ = D, then we 
read the datum A < B, the pr e e i ona relef edge ie deleted, only 
the daU edge remains 

We did not perform theoretically rig orou s derivations of confi- 
dence factor*. Thie ie beraaee the exact raise for combining confi- 
dences ie not important in our scheme. Confidences propagated with 
velocity rales have a •time-decay* built in. That ie, if A < £(*q);c/ : 
*» with appropriate vefodtks, then we derive A < B(ti)\cf : **, 
where x? < x. Con fi d en ce factor* are combined for transitivity rales 
by taking the min of all the relatione involved, then applying a decay 
factor (called a Spatial decay*) to the resulting number. 

We present the r e writ e of an expe rim ent in the series of figures 3 
- 7. The input ie echoed in the *in pot- list* window. Tbs * active- 
relations* window contains a list of edges with the label (reeeoo) end 
confidence. In figure 3 the data ie only the three v m otion s of the 
objects A , 3, and C. In figure 7, the data are A < C and B < A, the 
transitive closure is performed which derives B < C. Data relatione 
have confidences of 1.0. Transitive relatione, i.e., tc are decayed. In 
figure 7, the aame three relatione remain, because the velocity rale 
ridel infer them. The confidences have all decr e ased. Notice the 
confidence for the relation 5 < C, the relation originally derived 
through transitivity, ie Jess than that of the other two relatione orig- 
inally derived from data. In figure 7, we have in the data O < A\ 
note the new relation has confidence 1.0. In addition, we have de- 
rived through transitivity the new relation D < C. In thie figure 
notice that 3 < C actually has two edges. One ie a velocity rale 
edge with confidence 0.6, derived from previous B < C edge. The 
other is a transitivity edge derived from the edges you see present, 
9 < Aief : 0.8 and A < C\cf : 0.8. This did not happen in fig- 
ure 7 at time 8 b ec aus e of the nature of the spatial and temporal 
decay factors. For this experiment, the spatial decay is larger than 
the temporal decay. Figure 7 has the contradictory relation A < D 
just read in. In Sgure 7, only the maximum relation of the contra- 
dictory relations is printed out. The old A < D because of mLel is 
act drawn, though it will be propagated into the future. There are 
other ways of choosing a set of relations without cycles. For example 
one may add up the confidences on A < B relatione and on A > B 
relations, then choose the maximum of the two. 

T Conclusion, consequences and next steps 

In this paper we have described a dynamic scene analyse system 
which uses qualitative information, available with current computer 
vision abilities, to calculate relative depths between surfaces. The 
qualitative information required are motion toward or away from 
observer and occlude r-occiudee ordering. The system derives further 
relations from the data. Errors and inconsistencies are tolerated by 
requiring confidence factors to decay on each inference step. We have 
also described ua this paper our approach to r e pr esen ting histories of 
such qualitative values. 

This research has opened a number of questions. Among the 
more important is the problem of unng such qualitative calculations 
as control for other computer ▼vkmi processes, or se intial estimations 
for thoes iterative algorithms requiring them. We see this qualita- 
tive assessment ae capable cf providing a focus of attention when 
resources are limited and for making real-time dynamic scene analy- 
sis possible The integration of qualitative procedures with numencal 
ones is an interesting problem. 
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